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Abstract 

This paper addresses the problem of unsupervised soft bit error rate (BER) estimation for any communica- 
tions system, where no prior knowledge either about transmitted information bits, or the transceiver scheme is 
available. We show that the problem of BER estimation is equivalent to estimating the conditional probability 
■ density functions (pdf)s of soft channel/receiver outputs. Assuming that the receiver has no analytical model 

' of soft observations, we propose a non parametric Kernel-based pdf estimation technique, and show that the 



resulting BER estimator is asymptotically unbiased and point-wise consistent. We then introduce an iterative 



£ — ■ Stochastic Expectation Maximization (EM) algorithm for the estimation of both a priori and a posteriori 

g j probabilities of „,n smi „ed ^formation bits, and .he Cassation of soft observadons aeeord.ng ,o —eo 

bit values. These inputs serve in the iterative Kernel-based estimation procedure of conditional pdfs. We analyze 
the performance of the proposed unsupervised and non parametric BER estimator in the framework of a multiuser 

X 

code division multiple access (CDMA) system with single user detection, and show that attractive performance 
are achieved compared with conventional Monte Carlo (MC)-aided techniques. 
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I. Introduction 

Monte Carlo (MC) techniques are generally used to evaluate the bit error rate (BER) or block error 
rate (BLER) of digital communication systems. In [1], a tutorial exposition of different MC-aided 
techniques has been provided, with particular reference to four methods: i) Modified Monte Carlo 
simulation (i.e., importance sampling); ii) extreme value theory; iii) tail extrapolation; and iv) quasi- 
analytical method. The modified Monte Carlo technique is achieved by importance sampling which 
means that important events, i.e., errors, are artificially generated by biasing the noise process. At 
the end of the simulation, the error count must be properly unbiased [2]. The extreme value theory 
[3], assumes that the probability density function (pdf) can be approximated by exponential functions. 
The tail extrapolation method, which is a subclass of the extreme value theory technique, is based 
on the assumption that the tail region of the pdf can be described by a generalized exponential class. 
The quasi-analytical method combines noiseless simulation with analytical representation of the noise. 
Except for the MC method, all these techniques assume perfect knowledge of the type and/or the form 
of noise statistics. In [4], high order statistics (HOS) of the bit log-likelihood-ratio (LLR) are used 
to evaluate the error rate performance of turbo-like codes. In this case, the characteristic function of 
the bit LLR is estimated using its first cumulants, i.e., moments, and the pdf is deduced with the 
aid of the inverse Fourier transform (IFT). In [5], a BER estimation technique where the transmitter 
sends a fixed information bit value has been proposed. At the receiver side, the BER is computed 
by estimating the pdf of received soft channel/receiver outputs. This technique is called soft BER 
estimation since the BER is estimated using soft channel observations/outputs without requiring hard 
decisions about information bits 1 . Unfortunately, all these methods assume that the estimator perfectly 
knows transmitted data. 

In many practical communication systems, the BER is required to be on-line estimated in order to 
perform system-level functions such as scheduling, resource allocation, power control, or link adaptation 
where the transmission scheme is adapted to the channel conditions (See for instance [6]). Under this 
framework, the BER estimation problem becomes very challenging because of the following main 
reasons: 

• In MC-based techniques, the unknown transmitted information bit values are required for 
computing the BER estimate, while in practical communications systems the BER estimation 

'in soft BER estimation, soft observations are used instead of hard decisions. This provides reliable BER estimates and reduces the 
number of required samples/observations compared with hard decision-based BER estimation techniques [5]. 
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should be performed in an unsupervised fashion because the estimator has no information about 
transmitted data. 

• Most of practical communication channels are quasi-static block fading, i.e., constant over a 
certain block duration, and randomly changes from block to block. Therefore, in order to provide 
the transmitter with a reliable BER information feedback for a given block, i.e., channel state 
information (CSI), only the soft observations corresponding to the actual block have to be used 
for estimating the instantaneous BER. In the case of MC-based techniques, this results in unreliable 
or even wrong BER estimates because the number of observations/bits is not sufficient. 

• The knowledge of the transmitter scheme, channel and interference model, and receiver technique 
greatly impacts the reliability of the BER estimate. In other words, it is generally quite hard 
to derive analytical expressions of the bit error probability (BEP) when the system suffers from 
interference or in the case of non-linear receivers (e.g. iterative interference cancellation receivers). 

This paper was mainly motivated by the above considerations. We assume that the estimator has no 
knowledge either about transmitted data, or the transmitter/receiver scheme and the communication 
model. We focus on the problem of BER estimation in a completely unsupervised 2 fashion. To the 
best of the authors knowledge, this issue has not been addressed before in the literature. We first 
provide a problem formulation where we show that BER estimation is equivalent to estimating the 
pdfs of conditional soft observations corresponding to transmitted information bits. Then, we introduce 
a non parametric Gaussian Kernel-based pdf estimation technique where no analytical model of soft 
observations is assumed. We study the asymptotic behavior of the resulting BER estimator and show 
that it is unbiased and point- wise consistent. As pdf estimation requires the knowledge of both a priori 
probabilities of information bits and the classification of soft observations according to the +1 and 
— 1 values of transmitted bits, we introduce an iterative Stochastic Expectation Maximization (EM) 
algorithm for iteratively computing these parameters. We analyze the performance of the proposed 
unsupervised technique in the case of a multiuser code division multiple access (CDMA) system with 
conventional single-user detection. Performance comparison shows that the proposed estimator clearly 
outperforms supervised MC-aided estimation. Interestingly, reliable BER estimates are achieved even 
in the high signal to noise ratio (SNR) and using only a few number of soft observations. 

Throughout the paper, the following notation is used. The cardinality iV of set C is denoted N = \C\. 

2 The aim of unsupervised estimation is to estimate the BER based only on soft observations that serve for computing hard decisions 
about information bits without requiring any information about transmitted data and transceiver scheme/model. 
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When X is a random variable, E [X] and Var [X] denote the mathematical expectation and variance 
of X, respectively. When / is a second derivative function, (/)' (x) and (/)" (x) denote its first and 
second derivative at point x, respectively. sgn{.) denotes the sign of the argument, and ln(.) is the 
natural log function. P [.] is the probability of a given event, and superscript T denotes the transpose. 

The remainder of the paper is organized as follows. In Section II, we provide some preliminaries 
about the proposed BER estimator and detail soft output pdf estimation; In Section III, we introduce 
the Gaussian Kernel-based BER estimator and study its asymptotic behavior; Section IV details the 
Stochastic EM iterative algorithm; In Section V, we carry out performance evaluation; The paper is 
concluded in Section VI. The proofs of the results are provided in the Appendixes A-E. 

II. Preliminaries and Output PDF Estimation 

A. System Model 

We consider a general communication system where a sequence of N independent and identically 
distributed (i.i.d) information bits i<i<7v e ~~ 1} * s transmitted using any transmission scheme. 
The transmitter can use any type of channel coding, modulation, multiple access method (e.g., CDMA, 
orthogonal frequency division multiplexing (OFDM), etc.), and/or multiple input multiple output 
(MIMO) technique such as spatial multiplexing, or space-time coding. The communication channel 
can either be time-variant or invariant with either flat or frequency selective fading. At the receiver 
side, any reception technique is performed ranging from the very simple matched filter (MF) receiver 
to advanced reception schemes such as multiuser detection (MUD) [7], interference cancellation- 
aided turbo detection/equalization [8], and space-time turbo equalization (See for instance [9]-[12] 
and references therein). 

Let X denote the random variable (RV) corresponding to channel/receiver soft outputs, i.e., 
observations, and (Xi) 1<i<N be the sequence of the realizations of X where each Xi corresponds to 
the decision statistic that serves for the computation of hard decision hi about transmitted information 
bit b-i as hi = sgn{X i ). Note that X, L may contain any type of interference such as co-channel 
interference (CCI), intersymbol interference (ISI), and multiple antenna interference (MAI), etc... . 
We assume no knowledge either about the transmission scheme, or about the channel model and the 
receiver technique. Our main purpose is to estimate in an unsupervised fashion the BER at the output 
of the receiver. Let ir + and 7r_ denote the probability that the transmitted bit bi is equal to +1 and 
— 1, respectively, i.e., 
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7T+ 4p[6 i = +i], 

(1) 

_tt_ 4p[6 i = -i], 

where 7r + + 7r_ = 1. Note that 7r + and 7r_ are not known at the receiver. The soft outputs (Xi) 1<i<N 
are random variables having the same pdf fx(x). The BEP is then given by 



/0 p+oo 
(x) dx + it- / fx (x) dx, 
■oo ./O 



(2) 



where f° x {-) (respectively, (■)) is the conditional pdf of X such that 6, = +1 (respectively, 6, = — 1). 
/x-(x) is a mixture of the two conditional pdfs f b x {x) an d fx~( x ) an d can therefore be written as 

f x (x) = Tr + f x + (x)+n-f x -(x). (3) 

A general transmitter-receiver scheme with soft outputs (Xi) 1<i<N and hard decisions (6i)i<i<7v is 
depicted in Fig. 1. 

B. Brief Overview of the Proposed BER Estimator 

As it can be seen from the generic expression of the BEP p e in (2), the BER can be estimated using 
the two conditional pdfs fx~(-), fx~(')> an ^ the a priori probabilities n + and 7r_. Note that all these 
parameters are unknown, and are required to be estimated before computing the BER. The estimation 
of conditional pdfs f b x (•) and f h x (■) can be performed based on observations X 1 , • • ■ , X N , where only 
those corresponding to transmitted information bit value +1 (respectively, —1) are used to estimate 
f'x(') (respectively, f x r (-)). Unfortunately, the problem of classifying soft outputs X 1 ,--- ,X N 
according to transmitted information bit values is itself a detection problem. The unsupervised BER 
estimator we propose in this paper allows us to classify soft outputs X 1: ■ ■ • , X N and estimate both the 
conditional pdfs f b x {-) and f b x {-) and a priori probabilities 7r + and 7r_ in an iterative fashion using 
a posteriori probability (APP) values P[bi = +1 | Xi] and P[bi — -1 | X<\ V i — 1, • • • , N. 

Let T denote the number of iterations (index t = 1, • • • ,T). At each iteration t, all APP values 
are computed using a priori probabilities and pdf estimates obtained at the previous iteration t — 1. 
The resulting APPs allow us to update the estimates of a priori probabilities and classify soft outputs 
X 1: --- ,X N into two classes according to transmitted information bit values. The two conditional 
pdfs are then re-estimated. This process is repeated T times, and the BER is estimated with the aid 
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of parameter estimates obtained at the last iteration T. Note that, at the first iteration, since no prior 
information is available, the classification of soft outputs X i: ■ • ■ ,X N can be performed with respect 
to the sign of each observation JQ, while the initial values of a priori probabilities n + and 7r_ can be 
derived using this initial classification. The diagram of the proposed iterative BER estimation algorithm 
is depicted in Fig. 2. 

C. Output PDF Estimation 

Both conditional pdfs f b ^ (x) and f b x (x) depend on the communication channel model and receiver 
scheme. Therefore, it is extremely difficult or even infeasible to find out the exact parametric model of 
these distributions. In this subsection, we introduce a Kernel-based method [13]— [15] to estimate both 
pdfs (x) and f x (x). The proposed technique is non parametric, and only requires soft outputs for 
each class. 

Let us assume that we can classify the set C = {X 1: --- ,X N } into two classes (i.e., partitions) 
C + and C_, where C + (respectively, C_) contains the observed received soft output X; such that the 
corresponding transmitted bit is hi = +1 (respectively, hi = — 1). In Section IV, we will introduce a 
Stochastic EM-based algorithm that allows us to classify the sets C+ and C_. Let N + (respectively, 
N_) denote the cardinality of C + (respectively, i.e., N + = \C + \, and N_ = |C_|. 

Using the Kernel technique, it follows that each conditional distribution can be estimated using the 
elements of sets C + and C_ as, 



where Hn + (respectively, h N _) is a smoothing parameter which depends on the number of observed 
samples, i.e., N + (respectively, NJ). Function K(-) is any pdf (called the Kernel) assumed to be an 
even and regular (i.e., square integrated) function with zero mean and unit variance. In the following, 
we provide equations only in the case of class C + as those corresponding to C_ can easily be obtained 
by replacing "+" by 

The choice of the optimal smoothing parameter is very critical since it impacts the accuracy of the 
pdf estimate. In [16], [17], it has been shown that if h N+ tends towards when N + tends towards 
+oo, the estimator f b x N+ (x) is asymptotically unbiased. It has also been shown that if h N+ — > and 




(4) 




(5) 



N+h N+ — > +00 when N+ — > +00, then the MSE of the Kernel estimator tends towards zero. The 
optimal integrated mean squared error (IMSE)-based smoothing parameter h* N+ is given as [5], 

h% + = N + ~* (j (ft))'* (M (K)) +1 * , (6) 

where M (K) = f^K 2 (x)dx, and J(f'x ) — f-™{(fx~)"{ x )} 2 d x is the integrated square second 
derivative of the conditional pdf f h ^ . 

As it can be seen from (6), the computation of h* N unfortunately depends on the unknown pdf f h ^ . 
We therefore suggest to use the Gaussian Kernel whose parameter M(K) is given as [5] 

M(K) = (7) 

while J{f b x) can be expressed in the case of a Gaussian conditional distribution, i.e., f h £ ~ 

J\f(m+, a\), as o 

J (fx) = s-^r, (8) 

where m+, and a\ are the mean and variance of subset C+, respectively. Details regarding the derivation 
of (8) can be found in Appendix I. It follows from (6) that the optimal smoothing parameter in the 
case of a Gaussian Kernel, and a Gaussian conditional pdf is given as, 

III. BER Estimation 
A. Gaussian Kernel-Based BER Estimation 

In this subsection, we derive the expression of the BER estimate assuming Gaussian Kernel-based 
pdf estimator. Let 9 = N + , h N+ , tt_, N_, h N _). The value of 9 is unknown and is iteratively 
estimated by iteratively classifying soft outputs Xl, • • • ,X N into two classes. At the last iteration T, 
a reliable estimate of 9 is reached and the BER is computed using the obtained estimate. Let us recall 
the expression of the BEP (2). Replacing the two conditional pdfs by their Gaussian Kernel-based 
estimates (4) and (5), and given the value of 9, the BER estimate is simply computed as 

where Q(-) denotes the complementary unit cumulative Gaussian distribution, i.e., Q(x) = 
fj~°° — L= ex p ( y —t 2 /2)dt. Details regarding the derivation of (10) are provided in Appendix II. 
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B. Theoretical Analysis 

In this subsection, we study the MSE-based convergence of the proposed iterative BER estimator. 
The first theorem shows that the proposed estimator is asymptotically unbiased. 

Theorem 1: Assume that the two conditional pdfs, f h x and f b x , are second derivative functions, that 
hN+ — ► and h N _ — > as N — > +oo. Then the soft BER estimator p e<N given by (10) is asymptotically 
unbiased, i.e., 

lim E \p e N ] = p e . 

Proof: See Appendix III for the proof. ■ 
The following theorem shows that the variance of the proposed estimator also tends towards zero. 

Theorem 2: Assume that the two conditional pdfs, f h x and f b x , are second derivative functions, 
that h N+ — > and h N _ — > as N — > +oo. Then the variance of the soft BER estimator p e>N tends 
towards zero as N tends towards +oo, i.e., 

jv Um oo E[(p e ,Ar-E[p eiJV ]) 2 ] =0. 
Proof: See Appendix IV for the proof. ■ 
Using Theorems 1 and 2, it is easy to show that the proposed estimator is point-wise consistent, i.e., 
the MSE tends towards zero as the number of samples N tends towards +oo. This result can be 
formulated in the following corollary: 

Corollary 1: Assume that the two conditional pdfs, f b x and f h x , are second derivative functions, 
that h N+ — > and h N _ — > as N — > +oo. Then the MSE of the soft BER estimator p CtN tends towards 
zero as N tends towards +oo, i.e., 

lim E[(p e , N -p e ]) 2 ] =0. 
Proof: The proof of this corollary is straightforward. First of all, we write E [{p e ,N — Pe]) 2 ] = 
E [ip e ,N — E[j3 ei Ar]]) 2 ] + (E[p e> jv] — Pe]) 2 ■ Then, by noting that p £tN is asymptotically unbiased, i.e., 
(E [p £ ,n] —p e ) — ► 0, (see Theorem 1), and that the variance of p e>N tends towards as iV — > +oo (see 
Theorem 2), we get limAr^ +00 E [(p e ,N - Pe]) 2 ] =0. ■ 

IV. Stochastic EM-Based Parameter Estimation 

In this section, we introduce a Stochastic EM -based algorithm to iteratively classify soft outputs 
X 1: ■ ■ ■ ,X N into the two classes C + and C_, and estimate 9. The EM algorithm was first introduced 
by Dempster et. al. in [18]. It iteratively computes, with the aid of the estimation and maximization 
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steps, maximum likelihood (ML) estimates when the observations can be viewed as incomplete data 
that contain missing information (i.e., the unknown data we have to decide about). Stochastic EM 
presents a generalization of EM techniques by introducing a random rule in the EM algorithm. Some 
applications of Stochastic EM can be found in [19], [20]. In our case, as we want to estimate the 
BER in an unsupervised fashion, the incomplete data correspond to observations X 1: ■ • • ,X N , while 
the missing data are transmitted information bits b\, ■ ■ ■ ,b^. 

A. Estimation Step 

In the estimation step of iteration t, we estimate the APPs 



(*) 
,(*) 



±P[b l = +l\X l ^-% 
±P[b l = -l\X^-% 



(ID 



of unobserved information bits bu for % = 1, • • • , N, conditioned on observations X±, ■ ■ ■ ,X N and 
the estimate of 9 obtained at the maximization step of previous iteration t — 1. Using simple 

mathematical manipulations, we show that the likelihood probabilities of bit 6; at iteration t can be 
computed as, 



(*) 



r (*-D/> + 



(t-i) 



i)(*i) 



(*-!)(*)+* 



X,N 



(t-1) 



X,N { _ > 

Pi- ~ (*-i) f b + iv \TT(*-i) f b - I v -i ' 
x,n^ ' x,N y _ ' 



(12) 



,(*) 



Note that for the sake of notation simplicity, the iteration index is dropped from pdf estimates 



TO and f b xN{t _ r) (X,). 



B. Maximization Step 

At iteration t, the maximization step allows us to compute the estimate 6® based on conditional 
probabilities obtained at the estimation step of the same iteration t. The estimate 6® is obtained by 
maximizing the conditional expectation Q of the log-likelihood of the joint event at iteration t. 
Assuming independent soft observations X 1: • — ,X N , we can ^express Q (9^) as 



Q (6 {t) ) = E 



In mp^fei] \X ir --,X N 



N 



= 2. KM**' 
i=i 



(t) 



(13) 
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By invoking the fact that + = 1, the maximization of Q (6) leads to the new estimates of a 
priori probabilities at iteration t, 



7T. 



(t) _ 1 v^jv „(*) 




(14) 

A 7 " 

The proof of (14) is provided in Appendix V. 
C. Classification Step 

The remaining parameters N+\ hf$ , hJ$_ and the two conditional pdf estimates f b+ (t) (.) 
and / ~ (t) (.) depend on the outcome of the classification procedure of subsets C + and C_ at iteration 

X ^ N _ 

t. Therefore, this procedure should be carefully performed since it greatly impacts the reliability 
of both pdf estimates and a posteriori probabilities in subsequent iterations, and consequently the 
accuracy of the BER estimate. Given estimates f b ^ N (t-i) (•)> and /^(t-i) (•) available at 

iteration t, we can classify soft outputs X ± , ■ ■ ■ , X N according to joint probabilities P [X^ bi = +1], 
and P[Xi,bi = — 1]. Using Bayes rule, we can obtain subsets d+ and at iteration t as C+ 1 = 
Xi ■ 4~ 1] fl\^ M > ^f-^-n W), and C W = Cf. 



X,N^ 1)K lJ ~ J X,N«- 1) 

Unfortunately, this classification procedure will prevent some erroneous soft outputs (i.e., those 
positive soft outputs corresponding to transmitted information bit value — 1 and vise versa) from being 
exchanged in the course of iterations between subsets C+ and C_. In the following, we introduce a 
Stochastic EM-based algorithm that randomly performs the classification of soft outputs X 1 , - ■ ■ ,X N 
using APP values. This method relaxes the condition on the exchange of soft outputs between the two 
subsets C+ and C_. 

The Stochastic EM technique uses a random Bayesian rule. At iteration t, it combines likelihood 
probabilities (15) and realizations U^\--- ,U$ of a uniform random variable U defined over the 
interval [0, 1]. The classification of soft outputs is performed as follows, 

'd? ={X 1 : p%>Un}, 



Q(t) = g(t) 



Parameters and are simply obtained as 



C 



(f) 



, and N® 



»(*) 



(15) 



. The optimal smoothing 



parameters h$ and h$_ are computed either with the aid of (9) assuming Gaussian conditional pdfs, 
or by iteratively using the exact expressions of J(fx~) and J{f h x ) followed by (9). The two conditional 
pdfs are then estimated using (4) and (5). Note that a® and correspond to the standard-deviation 
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of subsets d+ and at iteration t. 

The proposed Stochastic EM-based unsupervised BER estimation algorithm can now be summarized 
as in Table I. 

V. Performance Evaluation 

A. Considered Framework 

To evaluate the performance of the proposed unsupervised and non parametric BER estimator, we 
consider the framework of a synchronous CDMA system with two users using binary phase-shift 
keying (BPSK) and operating over an additive white Gaussian noise (AWGN) channel. We restrict 
ourselves to the conventional single user CDMA detector. Performance assessment in the case of 
advanced signaling/receivers is not reported in this paper due to space limitation and is left for future 
contributions. 

With respect to the considered framework, the received L Sf xl chip-level signal vector at discrete 
time instant % can be expressed as 

v t = Ai&^si + A 2 bf } s 2 + n h (16) 

where L S f denotes the spreading factor, and s k e {±1/ V 'Lsf] Lsf is the spreading code corresponding 
to user k. A k is the amplitude of user k — 1, 2, bf^ is the information bit value e {±1} of user k at time 
instant i, and e R LsF is the temporally and spatially white Gaussian noise, i.e., ~ Af (0, a 2 I LsF ). 
The a priori probabilities of information bits are supposed to be identical for both users, i.e., n + = 



f) = +1 



, and 7r_ 



&<*> = -1 



V/c, i. 



The decision statistic that serves for detecting user 1 at time instant % is = s^r^ [7] and is 

given as, ri , m m 

Xf> = A^ + A 2 bfp + fif\ (17) 

where p is the normalized cross-correlation between the two spreading codes S\ and s 2 , and n- 1 ^ is 
the Gaussian noise at the output of the single user detector, i.e., n^p ~ A/"(0, a 2 ). The decision about 
information bit b^p corresponds to the sign of decision statistic x\ l \ i.e., b^ = sgn (^P) • Note that 
the soft output X^ in (17) contains a mixture of a Gaussian noise and a RV whose pdf is unknown 
at the receiver. Using (17), we can easily show that the BEP for user 1 is 

rt f A 1 -A 2 p \ , ( 2 , 2 , f Ai + A2p \ 

Pei = 27T + 7T_Q + (7T + + 7T_) Q . (18) 
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In the following, we use the two spreading codes 

1 



+ + + + j , (19) 

— + + — r, (20) 



where the cross -correlation is p = 0.4286. We consider the case where the two users have equal 
powers Ai = A 2 = I. The SNR at the output of the MF of each user is therefore SNR = l/2a 2 . Note 
that under these assumptions (cross-correlation value and equal powers), soft decisions X^~\ ■ ■ ■ , X$ 
will be corrupted by severe MAI. This is extremely challenging for the proposed BER estimator 
since parameter estimation (such as a priori probabilities) will be performed in the presence of strong 
interference. In all simulations, we consider T = 6 iterations for the Stochastic EM-based parameter 
estimation while at each iteration t = 1, • • • , 6, two iterations are used for computing the optimal 
smoothing parameters h* N and h* N _ as mentioned in Subsection IV-C. Note that for all the scenarios 
we consider in the following, we assume that the proposed unsupervised and non parametric BER 
estimator has no prior knowledge either about a priori probabilities n + and 7r_ or the classification of 
soft outputs X[ 1] ,--- ,X$. 



B. Numerical Results 

1) Performance for Uniform Sources: First, we consider the case of equiprobable information bits, 
i.e., (7r + = 7r_ = 1/2). The number of soft outputs that serve for estimating the BER is N = 10 4 
observations. In Fig. 3 and Fig. 4 we present both the theoretical and the estimated conditional pdfs 
(at the last iteration T = 6) for SNR = OdB and lOdB, respectively. We observe that the proposed 
BER estimator provides accurate estimates of conditional pdfs. Note that for low SNR (Fig. 3), the 
variance of the MAI plus thermal noise is very high, and therefore the set of soft outputs has a 
large definition support. This means that in order to achieve smooth pdf estimates, a large number of 
observations is required. This explains the oscillatory behavior we observe around +1 and —1 for low 
SNR = OdB. When the SNR is increased to SNR = lOdB, no oscillations are observed since the MAI 
plus noise variance is reduced and the definition support of soft observations becomes very tight. In 
Fig. 5, we provide the BER estimation performance. We notice that the proposed unsupervised and non 
parametric BER estimator offers the same performance as the MC-aided method where the estimator 
has perfect knowledge about the transmitted information bits, i.e., perfect classification of soft outputs 
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■ ■ ■ ,Xjp. Note that when the proposed estimator knows the transmitted information bits, then 
its supervised version corresponds only to the computation of the smoothing parameters followed by 
the BER estimation according to (10) and the iterative Stochastic EM algorithm is not required. 

2) Performance for Non Uniform Sources: We now turn to the case when the information bits 
are not equiprobable. We consider the scenario where n + = 0.75 and 7r_ = 0.25. The number of soft 
outputs is kept to N = 10 4 . In Fig. 6, we report both the theoretical and estimated weighted conditional 
pdfs for SNR = lOdB. A quick inspection of the performance graph shows that even if the a priori 
probabilities are not equal, the proposed technique provides reliable pdf estimates. Also, note that as 
in the previous scenario of equiprobable information bits, the oscillatory behavior is not observed. At 
the last iteration T = 6, the estimated a priori values are 7r + = 0.752 and 7r_ = 0.248. We therefore 
conclude that the proposed unsupervised and non parametric BER estimator achieves reliable estimates 
of conditional pdfs for high SNR independently of the distribution of information bits. 

3) Performance in the High SNR Region: We now focus on the behavior of the proposed BER 
estimator at the very high SNR region where it is difficult to achieve a reliable BER estimate when 
using MC-aided techniques with a limited number of soft observations. In Fig. 7, we report the BER 
estimation performance with n + = n_ = 1/2, and using only N = 10 3 soft observations. The proposed 
technique provides reliable BER estimates (with respect to the theoretical curve) for SNR values up 
to SNR = 16dB, while the MC technique fails to do so and stops at SNR = 8dB because of the very 
limited number of transmitted information bits. This presents a major strength of the proposed BER 
estimator, and is very promising for many practical systems where it is required to estimate the BER 
of the communication link in a real-time fashion based only on a few data frames. 

VI. Conclusions 

In this paper, we considered the problem of unsupervised BER estimation for any communication 
system using any modulation/coding or signal processing technique. We proposed a BER estimation 
algorithm where only soft observations that serve for computing hard decisions about information bits 
are used to estimate the BER, and no prior knowledge about transmitted information bits is required. 
First of all, we provided a formulation of the problem where we showed that BER estimation is 
equivalent to the estimation of conditional pdfs of soft observations (conditioned upon transmitted 
information bit values). We then proposed a BER computation technique using Gaussian Kernel- 
based pdf estimation. We provided a theoretical analysis of the estimator, and showed that it is 
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asymptotically unbiased, and point-wise consistent. Then, we introduced an iterative Stochastic EM 
technique to compute the parameters that serve for the estimation of conditional pdfs based only of 
soft observations. The proposed method involves the EM steps to estimate the a priori probabilities of 
transmitted information bits, and a Stochastic classification step to classify soft observations according 
to information bits. Finally, we evaluated the performance of the proposed unsupervised BER estimation 
technique in the framework of CDMA systems to corroborate the theoretical analysis. Interestingly, we 
showed that when classical MC methods fail to perform BER estimation in the region of high SNR, 
the proposed estimator provides reliable estimates using only few soft observations. 

Appendix I 
Proof of Equation (8) 

Let us compute the exact value of J(fx~) when the conditional pdf f b ^ is assumed to be Gaussian 
with mean m + and variance a\. We have, 



fx + {x) 



1 



cxp 



[x — m A 



/ 2na + ' \ 2a 2 4 

Then, the second derivative of the conditional Gaussian distribution /t + can be written as, 



(1.1) 



1 



cxp 



(x — m + Y 



_ (x - m+) 
2al 
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(1.2) 



'2;;r> :; \ 2al 

Using the following change of variable, t = V2(x — m + )/a+, we can write the squared second 
derivative integral of the conditional Gaussian pdf as, 

»+oo f . ... ^ 2 



j(f b x + : 



oo 

1 



fx) ( X ) \ dx . 



27T(7« 



+oo 



(-?) 


[,-£1 




2_ 



V2 



(1.3) 



Let m k denote the A;th moment of a zero-mean and unit-variance Gaussian distribution. Then with 
respect to (1.3), we get, 



Afx + ) 



i 



1 



1 



t z 

ex P 1 "IT 



2 t 
1 — t + — 

4 



dt, 



m - m 2 + 



7714 

~4~\ 



(1.4) 



As for a zero-mean and unit-variance Gaussian random variable, we have m = 1, m 2 — 1 and 
m 4 = 3ml = 3, we finally get, 



15 



(1.5) 



Appendix II 
Proof of Equation (10) 

By invoking the expression of the BEP (2) and the estimates f^~ N and f b XN _ of the two conditional 
pdfs and f b x , respectively, we can express the BER estimate as, 

f b X,N+ ( X ) dx + 71 - / /x>_ ( X ) dx ■ ( IL 1) 

■oo JO 



-v ' v- 

B+ B- 



Given the fact that the two conditional pdfs are estimated using Gaussian Kernels according to (4) and 

(respectively, t = ^ 



(5), and using the following change of variable, t = (respectively, t = for B + (respectively, 



BJ), we get, 



- V f 

N. 



h N + 



1 f% + 1 * 2 



j r+cc 1 

= — > / -=e~dt, 



J N.h N _ \ h N _ J 

1 /•+» 1 _ tl 

The BER estimate (10) is then obtained by combining (II. 1), (II.2), and (II.3). 
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Appendix III 

Proof of Theorem 1 
Let us recall that the proposed soft BER estimator is given by 

Pe,N = n + B + + it_B_, 



where, 



B 



+ 



J-ooN + h N+x ^ c+ \ h N+ J 



B_ 



+00 



x z eC- 



X - X 



h 



dx. 



(III.l) 



(m.2) 



(in.3) 



Let us examine the mathematical expectation of B + , i.e., E [£+]. With respect to (III. 2) we get, 



E[B, 



N+h 



-jV+E 



K 



x — X 



h 



N+ 



dx, 



—00 




N+h N+ 



ft, AT. 

rr — -u 



+ 



A ? 4 



f b £{u)du J rfx, 



(HI.4) 

(in.5) 

(III.6) 



where (III. 5) is obtained by noting that E 



K 



x-Xj 

flN, 



is identical for all X e C+ and can be evaluated 



only for a particular Xj e C + . Using the following change of variable t 



we have, 

ft AT , 



E[£. 



ftjV. 



— ( [ K(t) f^(x - h N+ t)dt] h N+ dx, 

N+ \J-00 J 

K(t) f b x + (x-h N+ t)dt] dx. 



■OO V_J_ 

/ /-+00 



—00 w— oo 



(III.7) 



As / x + is assumed to be a second derivative pdf function, we can write a Taylor series expansion of 
f b x as follows, 



2 +2 



#(* - h N+ t) = f b x + ( X ) - h N+t ( X ) + -»±- (f x + y ( X ) + o {h% + t 3 ) . dug) 

Then, combining (III.7) and (III. 8) we get, 



E[B. 



+J 



00 \ j —00 



K(t) 



f x +(x) - h N+ t (&y (x) + (f b x +y (x) 

+0 {h 3 N+ t 3 )] dt) dx, (IH.9) 
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/O / /-+oo , /-+oo 

#(*) / if (t) eft - f/i + ) / tK (t) dt 

-oo \ J —oo J— oo 

+ -f L (/x) 0*) ) f 2 tf(f)dtUz + 0(/&). (111.10) 



As if (.) is a zero mean and unit variance Gaussian Kernel, we get f_™ K (t) dt = 1, /_ ~ tif (t) rft = 
0, and f*™t 2 K(t)dt = 1. Exploiting these properties and the fact that J*^ (^)^x = 

(0)' (m.10) can be expressed as, 



E[B + ] = £j x ^x)dx + ^(f b x + )\o) + 0(h% + ). (III.ll) 
As h N+ — » when N — > +oo, we therefore get, 

/•O 

lim E[B+] = / f b x + (x)dx. (111.12) 

V-00 

Similarly, We can show that 

JV^+oo 

Combining (III.l), (111.12), and (111.13), and by invoking the BEP (2) we get, 



P + OO 

lim E[S_] = / fx(x)dx. (111.13) 



/0 p+oc 
fy(x)dx + 7i _ / f b x (x)dx 
oo JO 



N- 

= Pe- (HI.14) 



Appendix IV 
Proof of Theorem 2 

First, recall that the proposed soft BER estimator is given by, 

Pe , N = 7r + 73 + + 7r_£L, (IV.l) 



where, 



N+h N+ £-^ c+ J_ OQ V /iat + 



73_ 
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Let us introduce the following quantities, 



A i+ = I K ( X - Xi ) dx, where X t G C+, 



(IV.4) 



r+co / 

/„ n 



X — X,; 



h 



dx, where Xi G C_ 



As Xi, • • • , X w are independent, the variance of p ej Ar can be computed as 



Var\p e>N ] = irlVar(B + ) + n 2 _Var(B_), 

T1 Vari^ + ^-VariAiJ. 



N + h%_ 



N.h%_ 



From (IV.2) and (III. 11), we get, 



E[A i+ ] = h N+ E[B + ] 

= h N+ jf f b x + (x)dx + h f- (#)' (0) + h N+ 0(h% + ). 



— 



(IV.5) 



(IV.6) 



(IV.7) 



Now, to determine the analytical expression of (IV.6), we must calculate E ] . E [A 2 _] and E 
can be similarly deduced. Using (IV.4), we get, 



E[Al] = E 



We can easily show that for the chosen Gaussian Kernel, we have, 



K {^Xi\ K (v-Xi 



h 



N+ 
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N+ 



h N+ /y/2 I \V2h N 



(IV8) 



(IV9) 



Using (IV.8), (IV.9), and the following change of variable, (v,w) = {^r,x — y), we have 



E [<] 



E 
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o r o 



K | Xi T 1 K I 1 d.v di, ; \ e c+ 
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h N+ /V2j \V2h N+ 

— - — = | dv I Xi g Cj 



diu I Xi G C-i 



(IV. 10) 
(IV.ll) 
(IV. 12) 
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where (IV.12) is obtained by noting that K(-) is a pdf, and therefore f R K(w)dw = 1. It follows from 
(IV. 12) that 

E [A 2 i+ ] = V2h N+ £°° (j° K (j^^J <kj fx + («) du. (IV. 13) 

Using the following change of variable, t = , u ~f ^ , we get, 

E = 4 + / fif (t) ft (x + dt dx. (IV. 14) 

As is assumed to be a second derivative pdf, a Taylor series expansion of ft can be written as, 

/i + (*+^f) = /i + w+^f (/i + )'w+^(/i + )"w+o(tx + ). ov.15) 

From (IV. 14), and (IV. 15), and exploiting the fact that K (.) is a zero mean and unit variance Gaussian 
Kernel, we get 

= /4 + (p e+ + % (0) j + O (/& + ) . (IV.16) 

Using (IV.7) and (IV.16), we obtain 

1 ^[<]^ + |(*)' (0) G-^) 

((/i + )'(0)) 2 + ^O(^ + ). (IV.17) 



?4 + 



47V. 

As h,N + — > and iV + ps 7r + iV when V — > +oo, it follows that, 

Var \AU 

lim Ar * +J = 0. (IV. 18) 

JV-+oo N + h 2 N+ 

Var [A? 1 

Similarly, we can show that linijv- > +oc ^ ^ J = 0. By recalling the expression in (IV.6), we finally 
get, limAr^+oo Var \p ejN ] = 0. 
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Appendix V 
Proof of Equation (14) 

Let us consider the maximization of Q (9^) in (13) with respect to (w.r.t) + = 1. If we 
add a Lagrangian Multiplier, we get 

l («<") = t l» {'f^, (X,)} + l° (*)} } - A + *!? - l) . (V.l) 



, . . dL($W) , 9L(0(*)) .. , 

Setting the two derivatives — ^ft" an d — H?> to zero > we nn d 

dir . ' dw y _ ' 



It) _1 T N (t) 
(t) (*) 

By invoking the fact that vr^ + 7ri* } = 1, it follows from (V.2) that A = N. 



(V.2) 
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Fig. 1. General transmission scheme for any transmitter and receiver with soft outputs Xi, ■ ■ ■ , Xn and hard decisions b\, ■ ■ ■ , fejv. 
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Fig. 2. The diagram of the proposed iterative BER estimation algorithm. 
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TABLE I 

Summary of the Stochastic EM-Based Unsupervised BER Estimation Algorithm 



Initialization (t = 1) 

1.1. Classify soft outputs X\, ■ ■ ■ , according to their signs, i.e., 
and C« = {Xi : X, < 0}. 



1.2. 

1.3. 
1.4. 



Deduce and N {1 \ 



i.e., JVf = 

(i) 



(i) 



and N_ 



(i) 



Compute standard-deviations a+' and a_ corresponding to C+' and Ci ±; , respectively. 
Compute the smoothing parameters h$ and according to (9). 



(i) 



.(i) 



{X t : X t > 0}, 



^ and 



v 



(i) 



1.5. Deduce a priori probabilities as ir^' — N ^..^ „_ — N 

1.6. Compute the two conditional pdfs using (4), and (5). 
Parameter Estimation at Iteration t 

2.1. Estimation Step 



2.1.1. Estimate APPs and using (12). 



2.2. 
2.3. 



Maximization Step 

2.2.1. Compute a priori probabilities 7r^ and 7r^ using (14). 
Classification Step 



2.3.1. Classify subsets and C^l' using APP values pf^ and p^ J according to 



At) 



the random Bayesian rule (15). 



2.3.2. 



r (t) 



and = 



At) 



2.3.3. Compute the standard-deviations and ct 1 l ; corresponding to C+' and C^" 1 , resp. 

2.3.4. Compute the smoothing parameters tiff and h$_ according to (9). 

2.3.5. Compute the two conditional pdfs using (4), and (5). 
Parameter Estimation 

Compute the BER estimate using 6>( T ) and (10). 



,(*) 



At) 



At) 
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Estimated pdf (right) 
Estimated pdf (left) 
Theoretical pdf (right) 
Theoretical pdf (left) 




Fig. 3. Estimated conditional pdfs for n+ = 7r_ = 1/2, and SNR = OdB. 
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Estimated pdf (right) 
Estimated pdf (left) 
Theoretical pdf (right) 
Theoretical pdf (left) 




Fig. 4. Estimated conditional pdfs for n+ = 7r_ = 1/2, and SNR = lOdB. 
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Fig. 5. BER performance comparison when 7r+ = 7r_ = 1/2, and N = 10 4 soft observations. 
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Estimated pdf (right) 
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Fig. 6. Estimated conditional pdfs for n+ = 0.75, tt_ = 0.25, and SNR = lOdB. 
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Fig. 7. Behavior of the BER estimation at high SNR with n+ = 7r_ = 



1/2, and only N = 10 3 soft observations. 



